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Examiner's Remark 

Applicant's amendment, filed on 10/22/2009, has been received and entered into the file. 
Claims 1-20 and 22-23 are pending. The objection to the claim and the 112 2 nd paragraph 
rejections of claims 12-19 have been overcome in view of applicant's amendments/remarks and 
are hereby withdrawn. 

Regarding applicant's argument that the objection to the specification is for special terms, 
the examiner would like to point out that here is a section from MPEP 608.01(o): 

Note that examiners should ensure that the terms and phrases used in claims presented 
late in prosecution of the application (including claims amended via an examiner's amendment) 
find clear support or antecedent basis in the description so that the meaning of the terms in the 
claims may be ascertainable by reference to the description, see 37 CFR L25(d)(l). If the 
examiner determines that the claims presented late in prosecution do not comply with 37 CFR 
1.75 (d)(1), applicant will be required to make appropriate amendment to the description to 
provide clear support or antecedent basis for the terms appearing in the claims provided no new 
matter is introduced. 

Therefore both terms and phrases in the limitations should have antecedent basis in the 
specification. As far as terms are concerned, there are terms that are not described in the instant 
specification, for example, "inclusion of those region", "exclusion of those region", "included 
region", "excluded region", "potentially affected by a flash", and "not potentially affected by 
flash", etc.. Therefore the objection to the specification has not been overcome. 

Regarding applicant's argument that because each claim transforms a visual depiction of 
an image, the claims are statutory, the examiner would like to point out that, first of all, the 
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image must be that of a physical object, and, secondly, according to the 101 Guideline of the 
examiner's art unit, the transformed physical object must be displayed in order to pass the 
transformation test. In the claims that are rejected under 101, none of the limitations recites 
displaying the multi-channel image in which the red-eye has been removed. A digital image is 
not a visual depiction of a physical object unless it is displayed. Therefore the claims fail the 
transformation test. The 101 rejections have not been overcome. 

Regarding applicant's argument that the examiner is making illogical argument that 
because Benati and the applicant's disclosure are both directed to identifying red-eye, and 
because both methods apply thresholds to a luminance channel, that the applicant's claim 
limitations must therefore be disclosed by Benati, the examiner disagrees with applicant's 
allegation. The examiner does not make the rejection just based on the both inventions are 
directed to identifying red-eye, and both methods apply thresholds to a luminance channel. The 
examiner makes the rejection based on the cited sections of Benati with reasonable and broadest 
interpretation. Claim language is given its broadest reasonable interpretation. In re Morris, 127 
F.3d 1048 (Fed. Cir. 1997). The identifying of a flash mask of the claimed invention is or is 
obvious in view of a sub-step of Benati. After the step of applying luminance threshold in 
Benati, other steps are performed by Benati to track down the red-eye regions within the area 
isolated by the flash mask, the same as the claimed invention. 

Regarding applicant's argument that Benati' s threshold values will not meet this test 
given that each of the threshold values is targeted to identify the ultimate red-eye color, the 
examiner would like to point out again that the instant application also focus on potential red-eye 
regions (Page 4, lines 23-25 of the instant application), the same as that disclosed by Benati. 
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Both the teaching of Benati et al and the instant application is on red-eye correction. 
Furthermore, the instant application specifically teaches finding red-eye region (Page 4, lines 23- 
15). The applicant makes a remark that the argument by the examiner is factually incorrect or 
irrelevant and points to Figs. 4A-4E for support, the examiner disagrees. Figs. 4A-4E only show 
the result of thresholding. As described in the instant specification in the paragraph that 
described Figs. 4A-4E, M f represents the areas in the input image that may contain red-eye 
artifacts, therefore the rest of the processing may be restricted to the regions identified by Mf. 
Thus the teaching of the instant application and that of Benati are the trying to target re-eye 
directly. Whether the area is much larger is irrelevant since the areas of potential red-eye in 
Benati could be much larger also. 

Regarding applicant's argument that each of claims 1, 7, 12, and 23 recite "produce a 
flash mask characterized by the inclusion of those regions of said multi-channel image 
potentially affected by a flash, and the exclusion of those regions of said image not potentially 
affected by a flash, irrespective of whether an included or excluded region is within the 
boundaries of a person's face", the examiner would like to point out again that the instant 
application does not provide description on the above limitation. 

Regarding applicant's argument that though those threshold values of Benati, if applied 
separately would identify regions outside a person's face, the examiner would like to point out 
that the instant application also does the same (see Figs. 4A-4E). Regarding applicant's 
argument that the examiner has make no effort to show that the luminance threshold range of 
Benati, i.e., between 40 and 166 meets the applicant's claim limitation, the examiner would like 
to point out that neither does the teaching of the instant application. Why would T f in the instant 
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specification guarantee that it would meet the claimed limitation? In other words, the limitation 
as claimed and as described in the instant application is equivalent or obvious from the teaching 
of Benati. 

Therefore the 103 rejections have not been overcome. 

DETAILED ACTION 
Specification 

1 . The specification is objected to as failing to provide proper antecedent basis for the 
claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01(o). Correction of the 
following is required: claims 1, 7, recite "produce a flash mask characterized by the inclusion of 
those regions of said multi-channel image potentially affected by a flash, and the exclusion of 
those regions of said image not potentially affected by a flash, irrespective of whether an 
included or excluded region is within the boundaries of a person's face". There is no antecedent 
basis for the claimed subject matter. 

Claim Rejections - 35 USC §101 

2. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

Claims 1-20 and 22-23 are rejected under 35 U.S.C. 101 as not falling within one of the 
four statutory categories of invention. The Federal Circuit 1 , relying upon Supreme Court 
precedent 2 , has indicated that a statutory "process" under 35 U.S.C. 101 must (1) be tied to a 

1 In reBikki, 88 USPQ2d 1385 (Fed. Cir. 2008). 

2 Diamond v. Diehr, 450 U.S. 175, 184 (1981); Parker v. Flook, 437 U.S. 584, 588 n.9 (1978); Gottschalk v. 
Benson, 409 U.S. 63, 70 (1972); Cochrane v. Deener, 94 U.S. 780, 787-88 (1876). 
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particular machine or apparatus, or (2) transform a particular article to a different state or thing. 
This is referred to as the "machine or transformation test", whereby the recitation of a particular 
machine or transformation of an article must impose meaningful limits on the claim's scope to 
impart patent-eligibility (See Benson, 409 U.S. at 71-72), and the involvement of the machine or 
transformation in the claimed process must not merely be insignificant extra-solution activity 
(See Flook, 437 U.S. at 590"). While the instant claim(s) recite a series of steps or acts to be 
performed, the claim(s) neither transform an article nor positively tie to a particular machine that 
accomplishes the claimed method steps, and therefore do not qualify as a statutory process. 
Claim Rejections - 35 USC §103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 12 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over Benati 
et al (US Patent 5,748,764, cited by IDS). 

Regarding claim 12, Benati et al teach a method to identify sub-regions of a multi-channel image 
containing red-eye (col. 4, lines 21-31), said multi-channel image having at least a first channel 
and a second channel (Fig. 5, first channel is hue and second channel is saturation, col. 4, lines 
231-31), said method comprising: 

(a) identifying a sub-region of said image as containing a red-eye region based upon, at 
least in part, applying a first mask to said first channel, said first mask comparing a first statistic 
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of at least one pixel of said image to a first threshold (col. 4, lines 21-31, the first statistic is the 
hue value of a pixel, first threshold is either 700 or 1010, Fig. 6a, see also col. 3, lines 50-53, col. 
5, lines 17-27, col. 4, lines 18-27, col. 5, lines 43-67, col. 8, lines 40-67) to produce a flash mask 
characterized by the inclusion of those regions of said multi-channel image potentially affected 
by a flash, and the exclusion of those regions of said image not potentially affected by a flash, 
irrespective of whether an included or excluded region is within the boundaries of a person's face 
(110 in Fig. 2, 210 in Fig. 3, Figs. 9-11, col. 4, lines 21-45, the bit map is used to provide such 
flash mask in the same way as the instant application, Fig. 9, See Page 4 and 5 of the instant 
specification); and 

(b) applying a second mask to said second channel of said flash mask, said second mask 
comparing a second statistic of at least one pixel of said image to a second threshold, said second 
statistic being a different property than said first statistic (col. 4, lines 21-31, the second statistic 
is the saturation value of a pixel, the second threshold is either 65 or 256, the interpretation of 
statistic is as explained by the applicant in office action dated 7/7/2008, i.e., it can be pixel value, 
Fig. 6C), 

(c) removing the identified said red eye from said multi-channel image (300 in Fig. 2). 
However they do not explicitly teach the order of applying the first mask and then the second 
mask. But the steps of Benati et al on col. 4, lines 20-45 can be performed sequentially and the 
order does not matter, that is, first applying the threshold for luminance first, and then applying 
the threshold for saturation to the image. It is mainly a design choice. It is desirable to be 
flexible when designing the algorithm. Therefore it would have been obvious to one of the 
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ordinary skill in the art, at the time of invention, to apply the threshold for luminance first and 
apply the threshold for saturation to the image later. 

Regarding claim 14, Benati et al teach the method of claim 12 wherein said first threshold is 
different than said second threshold (col. 4, lines 21-31, the thresholds for hue and saturation are 
different). 

5. Claims 1-5, 7-11, 13, 15, 17 and 23 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Benati et al (US Patent 5,748,764, cited by IDS) in view of Koga et al (U S 
Patent 5,848,185). 

Regarding claims 1, 7 and 23, Benati et al teach a method to identify sub-regions of a multi- 
channel image (Figs. 1 and 2) comprising: 

converting and providing said multi-channel image to a modified multi-channel image 
wherein at least one of said channels is an enhanced luminance channel that has more than 60% 
of the luminance information of said multi-channel image (col. 3, lines 46-62, col. 4, lines 17- 
28, col. 5, lines 43-57, col. 8, lines 44-63, the lightness channel contains 100% luminance and 
therefore contains greater than 60% of the luminance information) and at least one of said 
channel is a saturation channel (col. 4, lines 17-28); and applying a threshold to said enhanced 
luminance channel (col. 4, lines 17-28), said threshold constructed to produce a flash mask 
characterized by the inclusion of those regions of said multi-channel image potentially affected 
by a flash, and the exclusion of those regions of said image not potentially affected by a flash, 
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irrespective of whether an included or excluded region is within the boundaries of a person's 
face (1 10 in Fig. 2, 210 in Fig. 3, Figs. 9-11, col. 4, lines 21-45, the bit map is used to provide 
such flash mask in the same way as the instant application, Fig. 9, See Page 4 and 5 of the 
instant specification) 

identifying a sub-region of said flash mask as containing the red-eye region based upon, 
at least in part, processing said saturation channel by applying a saturation mask to one or more 
pixels of said image (col. 4, lines 17-45); 

removing the identified said red eye from said multi-channel image (300 in Fig. 2). 
However they do not explicitly teach obtaining the flash mask first and then applying a 
saturation mask and said saturation mask comparing the standard deviation of the saturation 
value of a respective pixel to a threshold and they do not explicitly identifying location 
variations in said saturation based upon the standard deviation of the saturation value of pixels 
in said channel that substantially includes said saturation. 

But the steps of Benati et al on col. 4, lines 20-45 can be performed sequentially and the order 
does not matter, that is, first applying the threshold for luminance first, then obtaining a flash 
mask, and then applying the threshold for saturation to the pixels of the image. It is mainly a 
design choice. It is desirable to be flexible when designing the algorithm. Therefore it would 
have been obvious to one of the ordinary skill in the art, at the time of invention, to apply the 
threshold for luminance first and provide a flash mask and apply the threshold for saturation to 
the pixels of the image. 
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In the same field of image segmentation and object detection, Koga et al teach applying a 
saturation mask to one or more pixels of said image, said saturation mask comparing the 
standard deviation of the saturation value of a respective pixel to a threshold (col. 15, lines 34- 
41, the saturation mask compares the variance of the saturation value of a respective pixel to a 
threshold which is equivalent to comparing the standard deviation of the saturation value to a 
threshold because the variance is square of the standard deviation, Fig. 13, the respective pixel 
is a pixel in the image segment) to determine whether an image segment is monochromatic or 
color (Fig. 13). Koga et al also teach identifying location variations in said saturation based 
upon the standard deviation of the saturation value of pixels in said channel that substantially 
includes said saturation (col. 15, lines 22-41, the variance is the square of standard deviation, 
and variance of the image segment represents the location variations of standard deviation and 
variance in the image segment, Fig. 13). It is desirable to be efficient when detecting red-eye 
region by first focus on color region and detecting color region/segment. The method Koga et 
al is also a method to try in the method of Benati et al with predictable results (In re KSR v. 
Teleflex Inc). Therefore it would have been obvious to one of the ordinary skill in the art, at the 
time of invention, to use the method of Koga et al to detect whether an image segment is color 
or monochromatic first and then to find skin area and red eye area on the color area so that red- 
eye detection is more efficient. 

Regarding claim 2, Benati et al and Koga et al teach the method of claim 1 . Koga et al further 
teach wherein said standard deviation of said saturation value of a respective pixel is measured 
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relative to the mean saturation of pixels in a neighborhood local to said respective pixel (col. 15, 
lines 21-34, the neighborhood is the image segment). 

Regarding claims 3 and 8, Benati et al and Koga et al teach the method of claim 1 and claim 7. 
Benati et al further teach wherein said modified multi-channel image has hue, saturation, and 
intensity channels (col. 4, lines 17-28). 

Regarding claims 4 and 9, Benati et al and Koga et al teach the method of claim 3 and 8. Benati 
et al teach wherein said saturation channel represents the relative bandwidth of the visible output 
from a light source (col. 4, lines 17-28, the value of Sat by definition is the relative bandwidth of 
the visible output from a light source). 

Regarding claims 5 and 10, Benati et al and Koga et al teach the method of claim 4 and claim 9. 
Benati et al further teach wherein said hue is substantially the wavelength within the visible-light 
spectrum at which the energy output from a source is the greatest (col. 4, lines 17-28, inherent 
from the definition of hue). 

Regarding claim 11, Benati et al and Koga et al teach the method of claim 7. Benati et al further 
teach wherein each channel of said multi-channel image is processed differently to identify said 
sub-region of said image (col. 4, lines 17-28, the thresholds are different for different channels). 



Application/Control Number: 10/676,277 Page 12 

Art Unit: 2624 

Regarding claim 13, Benati et al teach the method of claim 12 where said first statistic is the 
intensity value of said pixel in said first channel. However they do not explicitly teach a second 
statistic is the standard deviation of a pixel in a second channel. In the field of object detection 
and extraction, Koga et al teach applying a saturation mask to one or more pixels of an image, 
said saturation mask comparing the standard deviation of the saturation value of a respective 
pixel to a threshold (col. 15, lines 34-41, the saturation mask compares the variance of the 
saturation value of a respective pixel to a threshold which is equivalent to comparing the 
standard deviation of the saturation value to a threshold because the variance is square of the 
standard deviation, Fig. 13, the respective pixel is a pixel in the image segment). It is desirable to 
be efficient when detecting red-eye region by first focus on color region. Therefore it would 
have been obvious to one of the ordinary skill in the art, at the time of invention, to use the 
method of Koga et al in the method of Benati et al to detect and extract color image segment for 
red-eye detection so that more efficient detection and extraction can be achieved. 

Regarding claim 15, Benati et al and Koga et al teach the method of claim 13. Koga et al further 
teach wherein said standard deviation of said saturation value of a respective pixel is measured 
relative to the mean saturation of pixels in a neighborhood local to said respective pixel (col. 15, 
lines 21-34, the neighborhood is the image segment). 

Regarding claim 17, Benati et al and Koga et al teach the method of claim 13. Koga et al teach 
wherein said second channel represents saturation (col. 15, lines 34-41). 
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6. Claim 18 is rejected under 35 U.S.C. 103(a) as being unpatentable over Benati et al in 
view of Liang et al (US Patent 6,678,413 Bl). 

Regarding claim 18, Benati et al teach the method of claim 17. However they do not teach the 
method comprising using a convex hull technique to identify contiguous regions. Liang et al 
teach a method comprising using a convex hull technique to identify contiguous regions when 
segmenting and identifying an object (col. 17, line 53-col. 18, line 6). It is desirable to 
represent and characterize an object by known techniques automatically (col. 3, lines 1-23 of 
Liang et al). Convex hull techniques are known to have the advantage of executing in linear 
time in a two-dimensional array as is usual in image processing. Therefore it would have been 
obvious to one of ordinary skill in the art, at the time of invention, to use the convex hull 
method of Liang et al to represent and identify contiguous regions in the method of Benati et al 
so that more efficient algorithm for red-eye detection can be developed. 

7. Claim 19 is rejected under 35 U.S.C. 103(a) as being unpatentable over Benati et al in 
view of Liang et al, further in view of Luo et al (US Patent 7,035,461). 

Regarding claim 19, Benati et al and Liang et al teach the method of claim 18. However they do 
not explicitly teach wherein contiguous regions having a size less than a threshold are removed 
as potential red-eye regions, said threshold computed dynamically based on the size of the input 
image. In the same field of endeavor, Luo et al teach resizing the input image (Fig. 12, col. 14, 
line 55-col. 15, line 11, col. 16, lines 46-59) and comparing the contiguous regions of the resized 
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image with a threshold and removing the regions having a size less than a threshold (col. 16, 
lines 7-14, Figs. 12-13). Depending on the size of the input image, the size of the red-eye is 
different also. Scaling the input image dynamically based on the size of the input image and then 
comparing the size of the contiguous regions with a threshold is equivalent to comparing the 
non-scaled contiguous region with a threshold that is dynamically computed based on the size of 
the input image. It is desirable to be efficient and correct when detecting red-eye pixels by 
eliminating pixels that are impossible to be red eyes (col. 1, lines 46-51 of Luo et al). Therefore 
it would have been obvious to one of ordinary skill in the art, at the time of invention, to use the 
method of Luo et al in the method of Benati and Liang ct al so that contiguous pixels are 
eliminated/removed as non red-eye pixels depending on the size of input image. 

Examiner's Comment 

8. Claims 6, 16, 20 and 22 cannot be rejected over the prior art. A statement of reasons that 
claims 6 and 16 cannot be rejected over the prior art is presented in the previous office action 
dated Oct. 23, 2007 and will not be repeated here. A statement of reasons that claims 20 and 22 
cannot be rejected over the prior art is provided in the office action dated 1/12/2009 and will not 
be repeated here. 

Conclusion 

9. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
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the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to YUZHEN GE whose telephone number is (571)272-7636. The 
examiner can normally be reached on 7:30am-4:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Matthew Bella can be reached on 571-272-7778. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Yuzhen Ge/ 

Primary Examiner, Art Unit 2624 



